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TITLE OF THE INVENTION 

DYNAMICALLY MODIFL\BLE KEYBOARD-STYLE INTERFACE 



BACKGROUND OF THE INVENTION 

Field of the Invention 

[0001] The present invention is directed to a dynamically modifiable keyboard- 
style interface, and, in one embodiment, to a laser drawn keyboard interface that 
dynamically changes according to user preferences or commands. 

Discussion of the Background 

[0002] Keyboards for personal computers, such as is shown in Figure 1, are 
known user-input devices. Known keyboards have included numerous keys, 
including a "standard" style keyboard utilizing approximately 101 keys. However, 
new design for keyboards have emerged that have required that an existing keyboard 
be thrown out and replaced by the new design since the physical arrangement of keys 
were such that new keys could not simply be added to an existing keyboard. 
[0003] Keyboards are also not the only user input device that a user often interacts 
with. In the laptop environment, such as is shown in Figure 2, a user also has access 
to a touch pad that sits between the user and the keyboard keys. Such a positioning of 
the mouse pad is preferable for manipulation of the mouse pad, but the mouse pad is 
often accidentally touched while typing. This causes the computer to erroneously 
believe that the user intended to signal a click or movement of the mouse. The 
positioning of the mouse pad also increases the distance that a user has to reach to get 
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to the keys, and increases the required depth of the computer in order to fit both the 
mouse pad and keys, 

[0004] Keyboards are also often bulky and sometimes require wires to connect the 
keyboard to the computer. Such requirements cause many users to wish to not carry a 
keyboard. Keyboards, however, are a more rapid input device than a PDA touch 
screen or character recognition solutions. Accordingly, many people would often like 
to have a keyboard without the hassle and bulk of carrying a keyboard. A known 
concept for a virtual keyboard, for computers and PDAs, has been presented by 
Canesta. The system includes a pattem projector that is believed to be fixed, an IR 
light source (behind an engraved film) and an IR sensor module. However, a problem 
associated with the design of the Canesta system is that, by virtue of the film used, the 
pattem drawn by the pattem projector and analyzed by the sensor module appears to 
be fixed and does not allow for dynamic reconfiguration of the drawn pattem and 
interactions with the pattem. 

[0005] Keyboards are also poor input devices in a multi-language environment. 
For example, in a kiosk in an international airport, it is difficult to have only one 
keyboard since keyboards are actually language dependent. For example, while the 
US-style keypad uses a "QWERTY" layout, France uses a "AZERTY" lay-out. Also, 
alternative keyboard interfaces (such as Dvorak style keyboards) exist, and users 
accustomed to those alternative interfaces may have difficulty in using a "Standard" 
keyboard. 

[0006] Some provisions exist to cause a computer to pretend that an existing 
keyboard with letters and symbols printed on it in one fashion is actually a keyboard 
corresponding to an alternate language. However, in such an environment, the user 
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does not actually see the letters as they would appear on the alternate keyboard, and 
the user can become confused. 

SUMMARY OF THE INVENTION 
[0007] The present invention is directed to a virtual user-input device that enables 
various input configurations to be utilized dynamically such that at least one of the 
keyboard layout and keyboard character mappings are changed dynamically. 
[0008] One embodiment of a system for achieving such a keyboard includes a 
dynamic pattern generation module and a motion sensor for determining interactions 
with the pattem(s) generated by the dynamic pattern generation module. The 
dynamic pattern generation module may be either a projector-based image or a 
monitor-based image. 

BRIEF DESCRIPTION OF THE DRAWINGS 
[0009] These and other advantages of the invention will become more apparent 
and more readily appreciated from the following detailed description of the exemplary 
embodiments of the invention taken in conjxmction with the accompanying drawings, 
where: 

[0010] Figure 1 is a schematic illustration of a known, fixed-key keyboard for a 
desktop-style personal computer; 

[001 1] Figure 2 is a schematic illustration of a known laptop-configvu'ation with a 
set of fixed keyboard keys and a touchpad area with corresponding mouse buttons; 
[0012] Figure 3 is a schematic illustration of a laptop including a keyboard 
implemented by a dynamic pattem generation module and a motion sensor according 
to the present invention; 
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[0013] Figure 4 is a schematic illustration of a laptop including a keyboard and 
mousepad implemented by a dynamic pattern generation module and a motion sensor 
according to the present invention; 

[0014] Figure 5 is a schematic illustration of an embodiment of a dynamic user 
interface based on a projection unit operating remotely from an application server 
from which the dynamic user interface is delivered over a wired network; 
[0015] Figure 6 is a schematic illustration of an embodiment of a dynamic user 
interface using a monitor-based image and operating remotely from an application 
server from which the dynamic user interface is delivered over a wired network; 
[0016] Figure 7 is a schematic illustration of an embodiment of a dynamic user 
interface based on a projection unit operating remotely from an application server 
from which the dynamic user interface is delivered over at least one wireless network; 
and 

[0017] Figure 8 is a schematic illustration of an embodiment of a dynamic user 
interface using a monitor-based image and operating remotely from an application 
server from which the dynamic user interface is delivered over at least one wireless 
network. 



DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 



[0018] According to an embodiment of a dynamic user interface according to the 
present invention. Figure 3 illustrates a laptop computer 200 including (1) a projection 
unit 300 at the top of the flip-top of the laptop computer 200 and (2) a motion sensor 
3 10 at the base of flip-top of the laptop computer 200. The projection vmit 300 may 
be a laser-based projection system and may include possibly at least one mirror. 



Potential laser-based displays include the laser-based display described in the article 
entitled "Cell phone captures and projects images" in Laser Focus World, July 2003. 
An alternate embodiment utilizes the Laser Projection Display from Symbol 
Technologies as is described in Symbol Technologies article "Preliminary Concept: 
Laser Projection Display (LPD)." The contents of both of those articles is 
incorporated herein by reference. However, the projection imit 300 is designed to 
display more than one pattem onto the base unit 320. The projection unit 300 may 
project a keyboard-style image 325 on either a planar surface or a 3D surface. The 
motion sensor 310 may include an IR beam transmitter and an IR sensor which may 
belong to either separate components or a single integrated component. Other 
technologies for motion sensing may also be used instead of IR transceivers. 
[0019] As shown in the example of Figures 3 and 4, a keyboard-style pattem 325 
is projected onto the base unit 320, As would be imderstood by those skilled in the art, 
the base unit 320 may include registration marks thereon in order to indicate to a user 
when the projection unit 300 is aligned with the motion sensor 310. Moreover, the 
laptop computer 200 may provide a calibration process or method by which points on 
the projected keyboard-style image 325 are identified to the motion sensor 310. This 
calibration process enables the image to be projected by the projection unit 300 even 
if the flip-top is not at the exact angle (compared to the base unit 320) for which the 
keyboard-style pattem 325 was originally created. 

[0020] In one embodiment of the present invention, shown in Figure 4, the 
projection unit 300 virtually superimposes or integrates with the keyboard-style image 
325 a mousepad 330 and corresponding mouse buttons 340. That is, a portion of the 
keyboard-style pattem 325 is virtually occluded and the mousepad 330 and buttons 
340 are drawn where a portion of the keyboard-style pattem 325 image otherwise 
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would have been drawn. (As would be understood by one of ordinary skill in the art, 
in embodiments utilizing a laser, the keyboard-style pattern 325 is not physically 
overwritten, but rather a portion of the keyboard-style pattem 325 is suppressed from 
being projected and the mousepad 330 and buttons 340 are projected in place of that 
portion of the keyboard-style pattem 325.) 

[0021] Alternatively, the keyboard-style pattem 325 can be created using 
technology other than a projection unit 300. For example, a fixed pattem can be 
printed onto the base unit 320. In such a configuration, the user would not be able to 
see any changes to the interface as it was dynamically updated, but various keyboard 
configurations could nonetheless be used dynamically. In addition, the base unit 320 
could be printed on with a variety of colors and patterns such that the user could, 
knowing the color corresponding to the current configuration, see several user 
interfaces simultaneously. 

[0022] In yet another embodiment, the "overhead" projection unit of Figures 3 
and 4 is replaced with a pattem generation device that is undemeath the surface of 
where the user is interacting with the interface. For example, an LCD panel or 
display is typed on and tracked by the motion sensor 310. In this configuration, no 
special touch sensitive material is required for the LCD since the motion sensor can 
use infrared to pick up the hand motions. Similarly, even a monitor (including flat 
panel monitors) under glass or other transparent material can be used to generate the 
keyboard-style pattem 325. The user simply types on the transparent material, 
protecting the monitor from harm while dynamically being able to update the display. 
As would be understood by those of ordinary skill in the art, in light of the 
inexpensive nature of computer monitors, several monitors may be used together to 
increase the size of the keyboard-style pattem 325 that can be interacted with. 



6 



[0023] As illustrated in Figure 5, the dynamic user interface may also be 
implemented in applications other than local computing environments (e.g., laptops 
and desktop computer environments). For example, a kiosk 250 may be equipped 
with a projection unit 300 that generates a user interface (e.g., a keyboard-style 
pattern 325 and/or a mousepad and corresponding buttons). Interactions with the 
keyboard-style pattem 325 are picked up by the motion sensor 310. Those 
interactions are communicated to a control and communications unit 350 over a 
commimications link 345. The control and communications unit 350 may then either 
process those interactions locally or send them on to an application server 400 
connected to the control and communications unit 350 by a LAN or WAN connection 
across at least one communications link 360. Such communications link may be any 
one or a combination of wired (e.g., Ethernet, ATM, FDDI, TokenRing) or wireless 
(e.g., 802.1 la, b, g and other follow-on standards, and other RF and JR standards) 
links. Moreover, the communication protocols may include any one of connection- 
oriented and connectionless communications protocols using either datagram or error- 
correcting protocols. Such protocols include TCP/IP, UDP, etc. 
[0024] Examples of applications for kiosks 250 include a public pay phone where 
the user interacts with the keyboard-style pattem 325 instead of a physical telephone 
interface. In light of the existence of a monitor and a keyboard-style pattem 325, a 
user of the kiosk 250 can also be provided with Intemet related services or enhanced 
calling features as well. For example, a user may browse emails or facsimiles 
corresponding to the user. In a kiosk that implements a phone booth, the kiosk may 
also include a phone handset or a speakerphone that the user utilizes to communicate 
with a called party. 
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[0025] In an environment where a kiosk provider does not want to incur the cost 
or risk of providing a projection unit 300, either a user can bring his/her own 
projection unit (e.g., integrated within a PDA or other portable device) or the kiosk 
provider can provide a base unit 320 with a predefined pattern printed thereon (see 
Figure 6). In an embodiment where the user brings his/her own projection unit, the 
projection xmit 300 may include an interface for receiving any one or a combination of 
power and control signals used to drive the projection unit. Such interfaces can be 
any wired or wireless communication devices including, but not limited to Ethernet, 
serial, USB, parallel, Bluetooth, CDMA, GSM/GPRS, etc. Control signals sent to the 
projection imit may include which one of plural user interfaces (or partial user 
interfaces) is currently projected. 

[0026] A provider of a kiosk 250 may also elect to utilize an under-mounted 
display technology, as described above with reference to a monitor or LCD panel 
covered with a transparent protective material. 

[0027] Figures 7 and 8 illustrate embodiments in which either the kiosk provider 
provides to a user or a user brings his own portable device 290 that interacts with a 
kiosk. The portable device 290 utilizes an RF module 380 (or an optical module such 
as an IR module) to conmiunicate with an application server 400 across a WAN/LAN 
using at least one commimications link 360. In this fashion, the entire control 
interface may be transported to and used in or near the kiosk. 

[0028] Other possible kiosks or applications can include any interface description 
accessible by the user that may be transmitted fi-om an application server 400 to a 
terminal containing the display mechanism, displayed on a surface, and operated upon 
by the user. Some applications include: web browsers; video conference 
applications (e.g., mute one or more participants, display a particular image to the 



8 



audience, control volume, dial-in participants); multimedia equipment controls (e.g., 
wave your hand down to decrease volume — ^up to increase it; dial a station, play a 
CD); information kiosks; advertising displays with feedback mechanisms; ticketing 
services; self-service interfaces (e.g., vending machines); remote device control (e.g., 
cameras, alarms, locks); remote vehicle control to, for example, control vehicles in 
hazardous environments; industrial environments (flow controls, 
heating/ventilation/air conditioning); clean rooms; sterile and medical environments 
where mechanical equipment placement is prohibitive; test equipment; hazardous 
environments; remote control of distant objects; e.g., factory equipment, defense 
applications, building security (alarms, cameras, locking mechanisms); and 
simulations. 

[0029] In order to dynamically generate the keyboard-style pattem 325 and/or 
determine a location on the keyboard-style pattem 325 that the user is interacting with, 
the present invention includes at least one of hardware and software for controlling at 
least one of the projection unit 300 and the motion sensor 310. In one software 
embodiment, a central processing unit (CPU) interacts with at least one memory (e.g., 
DRAM, ROM, EPROM, EEPROM, SRAM, SDRAM, and Flash RAM), and other 
optional special purpose logic devices (e.g., ASICs) or configurable logic devices 
(e.g., GAL and reprogrammable FPGA). The kiosk 250 may also include a floppy 
disk drive; other removable media devices (e.g., compact disc, tape, and removable 
magneto-optical media); and a hard disk, or other fixed, high density media drives, 
connected using an appropriate device bus (e.g., a SCSI bus, an Enhanced IDE bus, a 
Ultra DMA bus or a Serial ATA interface). The kiosk 250 may fiirther include a 
compact disc reader, a compact disc reader/writer unit or a compact disc jukebox . In 
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addition, a printer may also provides printed listings of work performed by a user at 
the kiosk 250. 

[0030] As stated above, the software for controlling the kiosk (or the kiosk and 
the portable device) includes at least one computer readable medium. Examples of 
computer readable media are compact discs, hard disks, floppy disks, tape, magneto- 
optical disks, PROMs (EPROM, EEPROM, Flash EPROM), DRAM, SRAM, 
SDRAM, etc. Stored on any one or on a combination of computer readable media, 
the present invention includes software for controlling both the hardware of the kiosk 
250 and for enabling the kiosk 250 to interact with a human user. Such software may 
include, but is not limited to, device drivers, operating systems and user applications, 
such as development tools. Together, the computer readable media and the software 
thereon form a computer program product of the present invention for providing a 
virtual user interface. The computer code devices of the present invention can be any 
interpreted or executable code mechanism, including but not limited to scripts, 
interpreters, dynamic link libraries, Java classes, and complete executable programs. 
Such software controls a pattern to be displayed to a user, and the pattern may be 
dynamically changed in response to configuration information provided to the 
software. Such changes include changes in keyboard key labels and positions and 
shapes of individual keys. Such software fiirther includes a dynamically configurable 
memory for determining which key corresponds to the portion of the keyboard 
interface that a user is interacting with. For example, if an existing key is split into 
two parts to make two keys in its place, a computer memory is updated to be able to 
differentiate interactions with one of the new keys fi^om interactions with the other of 
the two keys. Similarly, if keys are added where no keys existed before, the software 
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tracks the location and extent of the new key. Such tracking may also occur for a 
virtual mousepad and virtual mouse buttons. 

[0031] In addition, any of the functions described above in terms of software can 
instead be implemented in special-purpose hardware such as FPGAs, ASIC, PALs, 
GALs, etc. 

[0032] Numerous modifications of the above-teachings can be made by those of 
ordinary skill in the art without departing from the scope of protection afforded by the 
appended claims. 
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